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There is little debate that the most essential skill for 
academic success is the ability to read. It follows that 
students must learn to read before they can read to learn. 
Despite the broad acceptance of the importance of print 
literacy, many students remain below grade level in their 
reading skills. Although the 2005 results from the 
National Assessment for Educational Progress indicate 
that Grade 4 reading scores in Indiana were one scale 
score point above the national average (218 to 217), 70 
percent of Hoosier students had reading scores at or 
below a basic level of proficiency at this grade level. ^ 

To improve the literacy skills of students in preschool 
through Grade 2 who are at risk for academic failure due 
to poor reading skills, the Indiana Early Literacy Inter- 
vention Grant Program (ELIGP) was established in 
1997. The grant program was a component of a statewide 
literacy initiative advanced by the Indiana Department of 
Education (IDOE) and Dr. Suellen Reed, State Superin- 
tendent of Public Instruction. 

In this report, the Center for Evaluation and Education 
Policy (CEEP) reviews the history of the ELIGP, the 
research base on individual intervention models, and the 
impact of the grant program on student achievement. In 
light of this information, CEEP examines the central 
question, “Is the Indiana Early Intervention Grant 
Program Working?” 

OVERVIEW OF THE GRANT PROGRAM 

Prom its inception, the ELIGP has emphasized local 
school decision making in choosing literacy 
interventions that are funded by the state of Indiana 
through a competitive grant process. Schools interested 
in receiving ELIGP funds must complete a detailed 
application describing the early literacy intervention to 
be used and how it will be implemented; the applications 
are then reviewed and assessed on their proposed 
implementation. The grant also funds professional 



development activities for school personnel that are 
directly related to the literacy intervention. As a part of 
the grant application for the 2006-07 school year, 
applicants have also agreed to administer a fall and 
spring assessment to determine if program 
implementation is proceeding as planned, to identify any 
implementation challenges that may need to be 
addressed, and to submit student literacy achievement 
data for state compliance and evaluation. The Center for 
Evaluation and Education Policy (CEEP) has monitored 
and evaluated the ELIGP since the 1997-98 school year, 
and selected findings are discussed here. 

In the 2005-06 school year, the program served 
approximately 12,700 students in 307 schools. 
Representing 150 school districts, these schools shared 
$3.62 million in grants for early literacy intervention 
programs. Individual grants ranged from $4,530 for the 
Tri-Creek School Corporation to $254,050 for the 
Indianapolis Public Schools, and the median grant 
amount was $39,590.^ (See Eigure 1 for ELIGP allocated 
funds from the 1997-98 school year through the 2005-06 
school year.) 

During the 2005-06 school year, the three most common 
literacy intervention models implemented by schools 
receiving ELIGP funding were Reading Recovery, the 
Waterford Early Reading Program, and Literacy 
Collaborative. (See Table 1 for a comparison of these 
programs.) A variety of other interventions including 
Success for All, Lour Blocks, Even Start, and full-day 
kindergarten have also been implemented locally with 
ELIGP funds since the inception of the grant program. 
Additionally, during the grant application window for the 
2006-07 funding period, school corporations submitted 
requests for other intervention programs not commonly 
funded in previous grant cycles, including Scott 
Eoresman Early Reading Intervention (ERI), Voyager 
Passport, and Read Well. CEEP will summarize the 
components and results of these programs with its report 
issued at the end of the 2006-07 school year. 
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ELIGP Allocated Funds to Schools 1997-2006 
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Figure 1 based on Plucker, J.A., Simmons, A.B., & Ravert, R. (2005). Indiana's Early Literacy Inter- 
vention Grant Program 1997-2004. Bloomington, IN: Center for Evaluation & Education Policy, and 
IDOE Andrew Conway, personal communication, 4/21/2006. 
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INDICATORS OF SUCCESS 

Variations among the programs and the measures used to 
define success complicate the task of comparing 
individual literacy programs and the likelihood of 
success for each intervention. Nevertheless, there are 
indicators which can help identify success, including 
student achievement, grade retention rates, and special 
education referrals; measuring parental involvement and 
professional development activities may also aid in 
identifying success. 

ISTEP+ 

Improved test scores on the Indiana Statewide Testing 
for Educational Progress-Plus (ISTEP-t) suggest that 
student achievement in Indiana is increasing. CEEP has 
studied the relationship between grant funding and the 
ISTEPh- and has considered other studies that examine 
similar variables. Research shows moderate evidence of 
achievement gains among students in schools funded by 
the ELIGP. Specifically, Reading Recovery has shown 
promising results in schools facing high rates of poverty. 
In 2001, CEEP reported an increase of nearly three 
percentage points in the number of students in high- 



poverty schools that passed the English/language arts 
portion of ISTEPh- at Grade 3 when Reading Recovery 
had been implemented. Likewise, during 2001-02, 
schools that implemented Waterford had a higher mean 
ISTEPh- passing rate than schools that did not use 
Waterford.^ 

Student Retention 

If necessary, schools may retain students in their current 
grades until they have mastered the required literacy 
skills. However, retention may be a poor option for 
addressing individual academic needs, or it may be 
viewed as a sign of an inflexible educational 
environment. Therefore, a reduction in retention rates 
may indicate greater flexibility with respect to 
accommodating learning diversity and greater student 
mastery of grade level standards. CEEP found that most 
programs funded by ELIGP between 1997 and 2002 
were associated with a reduction in retention rates in the 
lower grade levels, although not all of the reductions 
were statistically significant. In particular, for the 2001- 
02 school year, ELIGP-funded schools had the lower 
retention rate of .83 percent in contrast to a rate of 1.27 
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TABLE 1. Comparison of the Primary Intervention Models Funded by the ELICP (2005-06 School Year) 





Waterford Early 
Reading Program 


Reading 

Recovery 


Literacy 

Collaborative 


Student 

Population 


Grades K-2 


lowest 20% of 
Grade 1 readers 


Grades K-6 

school-wide framework 


Organizational 

Features 


technology-based, 
classroom-based, 
ability grouping, 
diagnostic procedures, 
frequent assessment, 
small groups 


pullout program, 
one-on-one, 
grade limit, 
book canon, 
diagnostic procedures, 
frequent assessment 


classroom-based, 
book canon, 

literacy-rich environment, 
small groups, 
systematic learning, 
trade books, 
interactive learning 


Instructional 

Philosophy 


child-centered, 
developmental, 
phonological awareness 


child-centered, 
phonological awareness, 
whole language, 
student empowerment 


child-centered, 
reflective practice, 
student empowerment, 
thematic units, 
whole language 


Instructional 

Features 


emergent spelling, 
essays, 

predicting meaning, 
multisensory awareness, 
creative writing, 
workbooks 


creative writing, 
predicting meaning, 
multisensory activities, 
journals, 
paired reading, 
self-selected reading 


big books, 
creative writing, 
guided composition, 
multisensory activities, 
paired reading, 
story maps, 
reading aloud 


Literacy 

Outcomes 


emergent literacy, 
meaning-oriented comprehension, 
critical literacy 


emergent literacy, 
meaning-oriented comprehension, 
critical literacy 


composition, 

meaning-oriented comprehension, 
critical literacy 


Parental 

Involvement 


book distribution, 
paired reading, 
parent awareness, 
take-home activities 


paired reading, 
reading instruction training 


book distribution, 
parent awareness, 
reading instruction training 


Teacher 

Development 


certified specialist, 
in-service workshops, 
school site training, 
modeling with coaching 


certified specialist, 
ongoing meetings, 
networking, 
university training 


in-service workshop, 
teacher collaboration, 
peer observation, 
on-site facilitator 



Table 1 based on St. John, E.P., Loescher, S.A., & Bardzell, J.S. (2003). Improving reading and literacy in grades 1-5: A resource guide to 
research-based programs. Thousand Oaks, CA: Corwin Press. 



percent in schools without the use of ELIGP funds. 
Furthermore, Reading Recovery was found to have a 
positive effect in schools with a high poverty rate; those 
that implemented this program reported a retention rate 
of .65 percent, in comparison to a rate of 2.96 percent in 
high-poverty, non-intervention schools. Waterford was 
the only program implemented during this period that 
was associated with little or no reduction in retention 
rates.^ 

Special Education Referrals 

A decline in special education referral rates following the 
implementation of a literacy intervention may also signal 
a successful intervention. CEEP reports that during the 
first year of the ELIGP (1997-98), some early literacy 
interventions were associated with lower special 



education referrals, although the results were not 
statistically significant. However, the analysis of data 
from subsequent school years through 2000 showed that 
special education referrals were not reduced, no matter 
which literacy intervention had been implemented. As 
CEEP noted, the full impact of these programs on special 
education referrals may require long-term studies, rather 
than brief, yearly reports.^ For a contrasting outcome, a 
New York University study on the impact that Reading 
Recovery (RR) had on special education referrals in the 
New York City area over three school years found a 9 
percent rate of referral for students who completed RR. 
Over the same period, students who had not completed 
the intervention had a 14 percent rate of referral; the 5 
percentage points difference was statistically 
significant.^ 
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Additional Indicators 

Finally, other outcomes such as an increase in parental 
involvement and teacher professional development can 
be used to indicate successful programs. The nature of 
many intervention programs requires increased commu- 
nication between the school and the home, and/or profes- 
sional development to enhance teachers’ skills in literacy 
instruction. Therefore, ELIGP funding has supported 
both of these aims. For example, ELIGP schools are 
more likely to distribute books to households in need and 
to utilize parent volunteers. In addition, funding corre- 
sponds with a greater frequency of in-service profes- 
sional development for school personnel than for 
comparison schools. These professional development 
activities may be funded through ELIGP as long as they 
are tied directly to the literacy intervention program 
being implemented in the school.^ 

EVIDENCE-BASED RESEARCH 

As noted above, it is important that ELIGP funding is uti- 
lized to implement literacy interventions for which there 
is evidence of success based on legitimate research. In 
order to assess research that investigates the effective- 
ness of educational practice, including reading instruc- 
tion, the U.S. Department of Education (USDOE) 
distinguishes between studies that provide rigorous evi- 
dence of educational outcomes from those that do not. Of 
those studies which provide rigorous evidence, the 
USDOE classifies them as providing either strong or 
possible evidence. Strong evidence results from research 
that is based on well-designed and implemented random- 
ized control trials. Possible evidence may come from one 
of two sources; 1) studies that include randomized con- 
trols, but which are lacking in some other aspect of meth- 
odological design, and 2) studies that use a comparison- 
group design in which there is a strong match between 
the comparison and intervention groups in factors such 
as demographics and student achievement.^ 

An ethical hindrance to procuring strong evidence is that 
in completely randomized control studies, some children 
may not receive the help they need if they are denied 
assistance because they are selected to be in the control 
group that does not receive the intervention. 
Consequently, educational researchers may include in 
their studies randomly selected comparison groups of 
students not in need of an intervention, but who match 
the students receiving the intervention in a number of 
factors, as described above. 



RESEARCH REVIEW 

This review covers the three most commonly 
implemented literacy interventions in Indiana schools 
that received ELIGP awards during the 2005-06 school 
year. Euture CEEP reports will provide research review 
information on the new intervention programs funded 
during the 2006-07 school year such as Scott Eoresman 
Early Reading Intervention, Read Well, and Voyager 
Passport. The methodology for all of the following 
research, except that reported by the Ohio State 
University, falls within the USDOE guidelines for 
evidence-based research. The Literacy Collaborative 
data reported by the Ohio State University derive from a 
multi-year, nationwide literacy assessment of every 
student who participates in that program, but those 
students are not compared to students who are not in the 
program. Nevertheless, the data from Ohio State provide 
valuable evidence for the efficacy of Literacy 
Collaborative. 

Waterford Early Reading Program 

Unlike Reading Recovery and Literacy Collaborative, 
the Waterford Early Reading Program (WERP) is a tech- 
nology-based intervention. Although students participate 
in the intervention through the use of personal comput- 
ers, WERP can be implemented as a small group pro- 
gram. Given that WERP is technology-based, the 
teachers and administrators, in a study reported in the 
Reading & Writing Quarterly, carefully chose it because 
WERP corresponded closely to the reading curriculum 
already in place, and/or it could be used both at home and 
at school. 

The study lasted two years and included children from 
two consecutive Grade 1 classes in a rural Midwest 
school. Ethnic and socio-economic data were not avail- 
able for the students in the study, but the student popula- 
tion of the school as a whole was 97 percent white, and 
11 percent of the students were eligible for free or 
reduced-cost meal programs. Eorty-seven students com- 
prised the control group, which did not utilize the inter- 
vention, and 46 children were in the experimental group. 
The students in each of these groups were then classified 
into low-, moderate-, or high-performing groups, based 
upon initial testing. 

When the reading performance of all the children was 
tested at the end of Grade 2, researchers found that the 
students in the experimental group improved their scores 
to a greater degree than did the students in the control 
group. In particular, the test scores of the experimental 
low-performance students were equivalent to those of 
the moderate-performance students in the control group. 
Eurthermore, the students in the low-performing experi- 
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mental group experienced more overall improvement 
than did the experimental students in either the moder- 
ate- or high-performing groups. 

Other research, conducted by a team based at Buffalo 
(NY) State College, included over 100 urban children in 
16 classrooms. The study investigated the effects of 
WERP on kindergarten and Grade 1 students 
participating in the program in comparison to students in 
the same grade levels not in the program. At the time of 
the study, the school district was comprised of about 72 
percent minority students, and 67 percent of the students 
qualified for free lunch. Additionally, approximately 18 
percent of the district’s students had a disability and 
about 6 percent of the students were English language 
learners. The school district and Waterford provided 
training and ongoing technical support for all of the 
teachers participating in this study. The results of the 
study indicate that there were no statistically significant 
differences between the two groups in reading and 
writing achievement at the end of the school year. 
Instead, the researchers found that student achievement 
was significantly associated with a child’s entering level 
literacy skills and the type and amount of teacher 
interaction with the students. 

Reading Recovery 

In contrast to Literacy Collaborative and WERP, Read- 
ing Recovery (RR) is a pull-out program that generally 
serves a limited number of Grade 1 students — those who 
score in the lowest 20 percent of their class for reading 
skills. Of the three programs reviewed in this report, 
Reading Recovery (RR) is the only program usually con- 
ducted on an individual tutoring basis. 

A professor from the University of North Carolina, 
Greensboro conducted a two-year study on RR in the 
Southern U.S.; this study compared 62 RR students with 
54 students in another early literacy intervention and a 
control group of 58 students. Many of the students were 
transient, approximately 65 percent of them were minor- 
ity students, and about 75 percent were eligible for free 
or reduced lunch. The results indicated that students who 
received one or the other intervention performed signifi- 
cantly better on tests of word recognition and spelling at 
the end of Grade 1, and had lower rates of retention in 
Grade 1 than those in the control group. In addition, stu- 
dents who received the intervention were less likely to be 
retained at the end of Grade 2 and continued to receive 
higher scores on word recognition tests in Grade 2 than 

1 9 

their control-group peers. 

Another study addressing the effectiveness and 
efficiency of RR based out of Oakland University 
focused on 148 Grade 1 students from across the nation. 



Seventy-four at-risk students were randomly assigned to 
groups to receive the intervention during either the first 
or second half of the school year. These students were 
then compared with a total of 74 low- and high-average 
students from the same classrooms who did not receive 
the intervention. Among the four groups, minority 
student enrollment ranged from 50 to 62 percent, and 36 
to 60 percent of the students qualified for lunch 
assistance. At the end of the school year, the at-risk 
students who received RR during the first half of the 
school year scored between the low- and high-average 
students on ten different measures of literacy skills, and 
the at-risk students scored higher than the high-average 
students on one of those tests. Students who received the 
intervention during the second half of the school year 
scored between the low- and high-average students on 
three literacy skill measures, and outperformed the high- 
average students on three other literacy skill measures at 
the end of the school year.^^ 

Reading Recovery has also been shown to be effective as 
a diagnostic tool for identifying students who may have 
a learning disability. For example, in a study using data 
from New York University’s RR database, researchers 
determined that students who successfully completed the 
intervention required testing for learning disabilities and/ 
or were placed into special education less often than 
were students in a control group that did not receive the 
RR intervention at all, or than students who did not 
successfully complete RR. The study took place over a 
three-year period and included 4,124 Grade 1 students in 
11 New York City districts. A high percentage of both 
RR and control students were eligible for lunch 
assistance (84-89 percent, respectively) and most 
students were minority (83-86 percent, respectively). 
The researchers concluded that using RR as a diagnostic 
tool could help schools and districts save time, money, 
and long-term educational resources that otherwise 
might be misspent on children who do not need them.*^ 

Literacy Collaborative 

Literacy Collaborative (LC) is unique among the three 
interventions reviewed in this report in that it is a 
framework designed for use in grades K-6, it is school- 
wide, and its implementation is closely monitored by 
professionals in higher education. Based at Ohio State 
University and working with several other institutions of 
higher learning, the LC project conducts nationwide fall- 
to-fall assessments of every child in LC schools through 
the use of multiple reading and writing tests. The 
cumulative results of the assessments enable LC schools 
to observe trends over time so that they can evaluate 
curricula and teaching methodology. In part, the 2003 
multi-year report looked at trends of student achieve- 
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ment from 1996 to 2001 as measured by two well-known 
literacy assessments and used data from 33 schools that 
had been part of LC for at least four years with the same 
LC coordinator. These schools were representative of 
other LC schools in terms of ethnic and economic 
diversity; 13 of the schools had more than 50 percent 
minority students, 14 of the schools had more than 50 
percent Caucasian students, and 29 of the schools had 
more than 25 percent of their students qualified for free 
or reduced price lunches. 

The results showed that although the scores of entering 
kindergarten students for a measure of letter-sound 
relationships remained relatively constant over the five 
years, as the students progressed through Grade 1, their 
scores continually increased. Each successive year of 
Grade 1 students had higher scores than the Grade 1 
students before them, despite the fact that as kindergarten 
students, each class entered with relatively similar 
scores. This progression is evidence that the longer LC 
was implemented, the more improvement each 
successive group of students demonstrated. Likewise, 
another assessment of students’ total reading skills 
(including vocabulary and reading comprehension) in 
the fall of Grade 2 also increased from year to year. 

In an independent study. Literacy Collaborative (LC) 
was compared to three other early literacy interventions; 
Success for All, Developing Literacy First, and Building 
Essential Literacy. Participants in the study were 145 
Grade 1 students in four Boston area schools; the stu- 
dents were predominantly minorities (68-97 percent) and 
85 percent of them received free or reduced lunch. Each 
of the four interventions was assessed for its efficacy in 
helping students to develop six literacy skills. An assess- 
ment in the spring showed LC proved to be better than 
two of the other interventions in the development of four 
skills, and it was better than one other intervention for 
one skill. In addition, it yielded the highest average writ- 
ing score of all four interventions. The researchers sug- 
gested that the emphasis on writing pedagogy in LC 
teacher training led to more and better writing help for 
the students from their teachers. In addition, because LC 
incorporates the reading and spelling of high frequency 
words, children receiving this intervention may have had 
more vocabulary accessible to them during the writing 
process. 

Overall, this brief review of the research literature 
indicates that these early literacy interventions do have a 
positive impact on students’ literacy development. 
However, an intervention must be carefully chosen to 
match the population being served and then be properly 
implemented and maintained in order to maximize its 
potential benefits. 



GRANT PROGRAM UPDATES FOR 2006-07 

In order to improve literacy education and better utilize 
resources, several new features were incorporated into 
the ELIGP for the 2006-07 school year. For the first time 
since the grant’s inception, student literacy assessments 
for every intervention program at participating schools 
will be conducted to quantify change in student achieve- 
ment associated with participation in an ELIGP-funded 
early literacy intervention. CEEP’s analysis of assess- 
ment data in the fall of 2006 will provide baseline data 
which will be compared to data from a mid-year or 
spring assessment. Drawing on empirical data, school 
personnel will be able to identify the strengths of individ- 
ual interventions and the challenges to their successful 
implementation. In addition, these assessments can 
inform the judgments of school personnel about the effi- 
cacy of the interventions in promoting student literacy 
achievement. 

To help maintain consistency in the reporting and 
assessment of student achievement, the IDOE 
encouraged schools in 2006 to use DIBELS (Dynamic 
Indicators of Basic Early Literacy Skills) or the Indiana 
Reading Assessment. However, grant recipients under 
certain circumstances (and dependent on the intervention 
model implemented) will be permitted to use other pre- 
and post-assessments. Additionally, the IDOE set an 
earlier submission date for grant proposals, enabling the 
Department to notify schools of their funding awards 
well in advance of program implementation. The 
Department also re-emphasized that funding was 
available only for evidence-based intervention 
programs. Furthermore, literacy intervention models that 
are established as frameworks to guide instruction will 
be less likely to be funded than intervention programs 
that directly serve and meet the literacy needs of 
students. Finally, only professional development directly 
linked to a school’s early literacy intervention may be 
included in a grant proposal for possible ELIGP 
funding.*^ 
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CONCLUSIONS AND RECOMMENDATIONS 



Conclusion 

The Indiana Department of Education ejforts to retool 
and reinvigorate the ELIGP should improve program 
effectiveness and accountability. 

As a result of the required use of student literacy 
assessments, the Department will be able to provide 
tangible, quantitative data about the effects of the early 
literacy intervention programs implemented locally at a 
depth previously not available. This information should 
be used to inform program oversight, local 
implementation, and future grant award determinations, 
thereby helping to ensure long-term, comprehensive 
early literacy reforms in the state of Indiana. 

Recommendation 

The IDOE must guide funded programs to utilize either 
DIBELS or the Indiana Reading Assessment to assess 
student literacy skills and monitor compliance with par- 
ticipation. Emphasis on the timely submission of perfor- 
mance data to CEEP is essential to ensure a valid and 
reliable evaluation of the information to inform future 
program administration decisions. 

Once the performance data have been analyzed and 
summarized, the IDOE should use this information to 
further refine and improve program administration at the 
state level and implementation at the local level. The 
results from the fall and mid-year or spring midterm 
assessments should also be utilized to better match 
interventions with student populations. The IDOE 
should fund only programs that demonstrate improved 
student literacy and reading levels. 



Conclusion 

The IDOE is funding local requests for early literacy 
intervention programs that research supports as evi- 
dence-based. 

The research summarized in this report provides 
substantiation that there is, generally speaking, research 
and evidence supporting the primary programs funded 
by the ELIGP. However, grant application requests 
regularly exceed the availability of grant funds and 
dollars are being stretched to fund as many applications 
as possible. Thus, given current funding levels, the 
funded intervention programs are only reaching a 
fraction of the at-risk student population. 

Recommendation 

Given the current statistics for the state of Indiana 
regarding the reading achievement of students, CEEP 
finds that continued funding is both warranted and 
necessary in order to reach a far greater percentage of the 
student population that are at risk for reading failure. 
Therefore, the IDOE should seek increased funding of the 
ELIGP from federal, state, and local sources to scale-up 
the grant program. By supporting the development of 
ELIGP, the literacy needs of at-risk students may be 
addressed in the most effective ways possible. 
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